Math 70 Homework 2 Michael Downs. 2(w i b) 0 = (w i b) 0 = 0 = b = 1 n. w i b 2 = nb = = z iz i 2z iaˆd i + ˆd 2 i

Transcription

1 1. I show that minimizing n x i b aĉ i 2 with x i, b, a R n and a 2 = 1 is the same as maximizing the variance of Za. The problem here is finding an n-dimensional best fit line with direction vector a and displacement vector b to the n-dimensional points x i. In order to do that we have to find a, b, and each ĉ i that minimize the residual error from each x i to the line. Each ĉ i represents the projection of the point x i onto the line in R 1. I will first find b by holding a and each ĉ i fixed and letting w i = x i aĉ i. We then wish to minimize n w i b 2 with respect to b: ) b w i b 2 = = ( b w i b 2 2(w i b) Setting equal to 0 and solving for b: 0 = 0 = 0 = b = nb = b = 1 n 2(w i b) (w i b) w i w i w i w i b So b = 1 n n w i = w = x a c. Let z i = x i x and ˆd i = ĉ i c. We must now find ˆd i such that each z i aˆd i 2 is minimized. z i aˆd i 2 = (z i aˆd i ) (z i aˆd i ) = z iz i 2z iaˆd i + ˆd 2 i Taking the derivative: ˆd i (z iz i 2z iaˆd i + ˆd 2 i ) = 2z ia + 2ˆd i 1

2 Setting equal to 0 and solving we get that ˆd i = z ia. Returning to the problem of minimizing n z i aˆd i 2 = n z i a(z ia) 2 : z i a(z ia) 2 = = = (z iz i 2z ia(z ia) + (z ia) 2 ) (z iz i (z ia) 2 ) z iz i (z ia) 2 n (z ia) 2 is a sum of positive terms so in order to minimize n z i a(z ia) 2 we have to maximize n (z ia) 2 = n ((x i x) a) 2 which is the same as maximizing the variance of Za. 2. Let V be the matrix whose columns are the (normalized) eigenvectors of Z Z in decreasing order. Then the first column of ZV is the projection vector Za. Then we can rewrite Z via singular value decomposition and use the fact that V is orthogonal to get: ZV = UΣV V = UΣ The matrix multiplication UΣ scales each column of U by the eigenvalue in the corresponding column of Σ. Thus Za is proportional to the first column of U. 3. Code used: 1 X = read. csv ( / Users / Michael / Desktop /R/ stud. perform. csv, header=t) #read in data n=nrow (X) ;m=ncol (X) 3 X = as. matrix (X[ 1 : n, 2 :m] ) # get r i d o f f i r s t column which numbers students 5 # compute Z m = as. matrix ( colmeans (X) ) 7 one = as. matrix ( rep ( 1, n ) ) Z = X one% %t (m) 9 tzz = t (Z)% %Z 11 # get a EG = e i g e n ( tzz, symmetric=t) 13 a = EG$ v e c t o r s [, 1 ] val = EG$ v a l u e s 15 p r o j=as. v e c t o r (X% %a ) 17 #p l o t p l o t ( proj, rep ( 1, n ), xlab= PCA p r o j e c t i o n s, ylab= ) 2

3 19 t i t l e ( P r o j e c t i o n on the maximum e i g e n v e c t o r ) 21 # get 5 best which ( p r o j %in% s o r t ( p r o j ) [ 1 : 5 ] ) 23 #q u a l i t y o f p r o j e c t i o n onto l i n e : 25 val [ 1 ] /sum( val ) which outputs (the proper order should be ): > hw2(3) [1] "The five best students: " [1] [1] "Quality of projection onto line: " [1] and the plot: Projection on the maximum eigenvector PCA projections 4. Code: 1 X=as. v e c t o r ( scan ( / Users / Michael / Desktop /R/ comwebhits. dat ) ) p l o t. e c d f (X, do. p o i n t s = FALSE, v e r t i c a l s = TRUE, xlab = Time o f the website hit, hour, ylab = P r o b a b i l i t y, cdf ) 3 # Get the median 5 med = X[ round ( l e n g t h (X) / 2) ] 7 #check f = e c d f (X) 9 p r i n t ( median : ) 3

4 p r i n t (med) 11 p r i n t ( e c d f (med) : ) p r i n t ( f (med) ) 13 # p r o p o r t i o n o f h i t s a f t e r 8 pm (20 in m i l i t a r y time ) : 15 p r i n t ( p o r t i o n o f h i t s a f t e r 8 pm: ) p r i n t (1 f (20) ) which outputs: > hw2(4) Read 100 items [1] "median: " [1] [1] "ecdf(med): " [1] 0.5 [1] "portion of hits after 8 pm: " [1] 0.06 and the plot: ecdf(x) Probability, cdf Time of the website hit, hour 5. Using R to draw n observations from U(0, 1). Code (this is a segment from the main hw2 file, the parameter n is passed into that function): # n s o r t e d o b s e r v a t i o n s from U( 0, 1 ) 2 obs = s o r t ( r u n i f ( n ) ) 4 # p l o t e m p i r i c a l cdf 4

5 6 p l o t. e c d f ( obs, do. p o i n t s = FALSE, v e r t i c a l s = TRUE, xlab = x, ylab = P r o b a b i l i t y, c o l= red, main = paste ( e c d f ( x ) vs cdf ( x ) f o r, t o S t r i n g ( n ), o b s e r v a t i o n s ) ) # p l o t the t h e o r e t i c a l cdf 8 x = seq ( from =.2, to = 1. 2, by =. 0 1 ) l i n e s ( x, p u nif ( x ), l, c o l= green ) 10 legend ( bottomright, c ( e m p i r i c a l, t h e o r e t i c a l ), l t y =1, c o l=c ( red, green ), bty= n, cex =.75) Plotting n = 10 and n = 1000: > par(mfrow=c(1,2)) > hw2(5,10) > hw2(5,1000) outputs: ecdf(x) vs cdf(x) for 10 observationsecdf(x) vs cdf(x) for 1000 observation Probability empirical theoretical Probability empirical theoretical x x Entire hw2.r: 1 hw2=f u n c t i o n ( problem =3, n=1000) { 3 dump( / Users / Michael / Desktop /R/hw2. r ) i f ( problem==3) 5 { X = read. csv ( / Users / Michael / Desktop /R/ stud. perform. csv, header=t) #read in data 7 n=nrow (X) ;m=ncol (X) X = as. matrix (X[ 1 : n, 2 :m] ) # get r i d o f f i r s t column which numbers students 5

6 9 # compute Z 11 m = as. matrix ( colmeans (X) ) one = as. matrix ( rep ( 1, n ) ) 13 Z = X one% %t (m) tzz = t (Z)% %Z 15 # get a 17 EG = e i g e n ( tzz, symmetric=t) a = EG$ v e c t o r s [, 1 ] 19 val = EG$ v a l u e s p r o j=as. v e c t o r (X% %a ) 21 #p l o t 23 p l o t ( proj, rep ( 1, n ), xlab= PCA p r o j e c t i o n s, ylab= ) t i t l e ( P r o j e c t i o n on the maximum e i g e n v e c t o r ) 25 # get 5 best 27 p r i n t ( The f i v e best s tudents : ) p r i n t ( which ( p r o j %in% s o r t ( p r o j ) [ 1 : 5 ] ) ) 29 #q u a l i t y o f p r o j e c t i o n onto l i n e : 31 p r i n t ( Quality o f p r o j e c t i o n onto l i n e : ) p r i n t ( val [ 1 ] /sum( val ) ) 33 } i f ( problem==4) 35 { X=as. v e c t o r ( scan ( / Users / Michael / Desktop /R/ comwebhits. dat ) ) 37 p l o t. e c d f (X, do. p o i n t s = FALSE, v e r t i c a l s = TRUE, xlab = Time o f the website hit, hour, ylab = P r o b a b i l i t y, cdf ) 39 # Get the median med = X[ round ( l e n g t h (X) / 2) ] 41 #check 43 f = e c d f (X) p r i n t ( median : ) 45 p r i n t (med) p r i n t ( e c d f (med) : ) 47 p r i n t ( f (med) ) 49 # p r oportion o f h i t s a f t e r 8 pm (20 in m i l i t a r y time ) : p r i n t ( p o r t i o n o f h i t s a f t e r 8 pm: ) 51 p r i n t (1 f (20) ) } 53 i f ( problem==5) { 55 # n s o r t e d o b s e r v a t i o n s from U( 0, 1 ) obs = s o r t ( r u n i f ( n ) ) 57 # p l o t e m p i r i c a l cdf 59 p l o t. e c d f ( obs, xlim=c ( 0.2,1.2), ylim=c ( 0, 1 ), do. p o i n t s = FALSE, v e r t i c a l s = TRUE, xlab = x, ylab = P r o b a b i l i t y, c o l= red, main = paste ( e c d f ( x ) vs cdf ( x ) f o r, t o S t r i n g ( n ), o b s e r v a t i o n s ) ) 6

7 61 # p l o t the t h e o r e t i c a l cdf x = seq ( from =.2, to = 1. 2, by =. 0 1 ) 63 l i n e s ( x, p unif ( x ), l, c o l= green ) 65 legend ( bottomright, c ( e m p i r i c a l, t h e o r e t i c a l ), l t y =1, c o l=c ( red, green ), bty= n, cex =.75) } 67 } 7